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Abstract: This session focuses on the challenges of using photographic evidence while conducting 
evaluation and research. The viability of photo-interviewing is discussed in terms of evaluating a 
technology infusion program in higher education. Offering promise in bridging physical and psy- 
chological realities, this study applied a photoanalysis model to clarify and extend oral interview 
data gathered from a hypermedia training program. Photo-interviews were conducted with a sample 
of 19 program participants. Four polemics concerning photo-interviewing as an inquiry tool are 
presented. 
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INTRODUCTION 

There is a duality to subjectivity in the social sciences that is often overlooked In education, 
and tnuning, for example, researchers tiy to keep the data clean and avoid projecting subjective 
feelings into the objective record. On the other hand, those engaged in needs assessments and 
process evaluations (especially those that are internally generated) must dance around a double 
edged sword particulariy when obj«:tives aie not clearly defined. Even those programs that have 
clearly defined obj-^dves in the original prt^josal often suffer from the consequences of 
predetermination. Particularly true when a program is in its first year of operation, rcductionistic 
and objectives-based data collection strategies are prone to generate insufficient data for ongoing 
program imr jvement 

Photography as a method of inquiry has rarely been applied systematically in educational 
program evaluation or research (Wachtman, 1978; Brown, et.al., 1982; Fang, 1985; English, 
1988). The viability of this method has been well documented in anthropology and sociology (e.g., 
Becker, 1978; Byers, 1964; Collier & Collier, 1986; and Garfinkel. 1967) and has been funhei 
illuminated by Susan Sontag's (1977) cautions. To unlock the meaning of this infomiation, one 
must examine more tiian empirical facts, It is necessary also to grapple with the problem of 
gathering subjective and intersubjective perceptions. 

This session will focus on some of the challenges of using photographic evidence and 
photo-interviewing in evaluation or research. Specifically, the authors will discuss the viability o! 
photo-iriterviewing as part of a perception-based model of evaluation to describe and judge a 
technology infusion program in higher education. Central to our investigation is the study of what 
Erickson and Mohatt (1982) call "imm^iiate environments of learning", namely an analysis of how 
instructional phenomena are socially and psychologically constituted. Photo-interviewing was 
selected as a strategy because it offers much promise in bridging two realities: the physical and the 
psychological. 

THE SETTING 

Recently, our College of Education began to operate a three year grant from Apple 
Computers Inc. A software tool, HyperCard, was selected to integrate many of die college's short 
and long-term technology goals. HyperCard was chosen as a method of integration based on two 
assumptions. Hrst, supplying faculty witii computers would not assure that they would become 
part of the educational and instructional program. For a variety of reasons, including a limited 
availability of computers, it was assumed faculty often would be reluctant to alter their teaching 
approaches. Second, HyperCard would provide a unique opportunity to involve faculty and 
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Students in the use of computers because of its relative ease of learning, reasonable cost, and 
potential for aIn:K>st imn^diate benefits. To encourage the use of HyperCard, ongoing intensive 
training classes wene made available (and continue to be available) to faculty and smdents. 

Given the innovative and evolutionary mmc of this project, standard objectives- based and 
management models of program evaluation were determined to be insufficienL A perception based 
model of evaluation (Kunkel & Tucker, 1983) incorporating many of Stake's (1982) tenets was 
adopted. During April of 1989, m^jor evaluation questions as well as standards of quality were 
negotiated between the mteraal evaluators, project staff, dea i of the college of education, and the 
funding agency's project officer. Using tfie perception based model, six major questions were 
negotiated which would capture three phases of the program: contexts, processes and products. 

MAJOR E VALUATK)N QUESTIONS ~ j 

1 . Who are the participants and what is the nature of their participation over time? i 

2. What training occurs and how effective is it? 

3 . What use of the MzKintosh o«:iu^ in classrooms of participating faculty? 

4. What other training is taken by faculty participants? \ 

5 . What is the resultant impact of the Macintosh's use on students? j 

6. What are outcomes, needed follow-ups, and replication strategies? 

Diverse sources of data, both qualitative and quantitative, were used to describe each 
question. Data sources included: 

• review of project proposals, project manager files ?.s well as curricular and training 
materials; 

• analysis of internal evaluation data such as benchmarics. context and summative questionnaires; 

• visitation to field applications throughout the year, 

• individual and group interviewing with representative samples of university staff, 
administrators, K-12 teachers, Apple program officers, and teacher education students. 

ADDING AN INNOVATIVE DATA SOURCE 

The Apple project was still evolving during its first year of operation. As a result, the 
evaluation took on a decided process-oriented cast and needed to be able to accommodate a certain 
amount of uncertainty. The authors felt photo-interviewing was a flexible approach which could 
collect unpredictable as well as scheduled behaviors in training. As Hall (1974) had advised, we 
approached the photographic content with an open strategy in order to respond to holistic content. 
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THE EPISODE 

This paper presents one application or "episode" of photo-interviewing. The episode 
involved an intensive workshqp entitled "Introduction to HyperCard". The workshop was 
conducted in a university computer lab with 27 Apple Macintoshes. There was one instmctor, one 
graduate assistant, aiKi 27 students. Chie data collector was a very experienced evaluator, but was 
less experienced with the technical aspects of photography. For the second data collector, the 
reverse was true. 

The woiicshop took place on two separate weekends. Each weekend began with a four hour 
session and continued with day-long sessions on Saturday and Sunday. On the fint weekend of 
the workshop, two photographs (with different lenses) were taken at i^jproximately twenty minute 
intervals by the two data collectors. After a review of the products of the first weekend's effons 
and subs^uent discussion between the evaluatcsrs, this collectbn strategy was adjusted. There 
were two reasons for this deliberate decision. First, artificially i*nposing an exact twenty minute 
interval between the photographs seemed contrived and inefficient Events, judged by the 
evaluators to be salient parts of the episode, were being missed. A second reason was that the 
evaluators, hesitantly at first, changed roles from "disiipUned" observers to observer-participants. 
Consequently, on the second weekend, photographs were taken which, from the perspective of the 
photographer/data collectors, functioned as a journal of the occurrences of the workshop. 

Hie photo-interviewing application occurred one month after the workshop. Attempts were 
made to schedule all trainees into small groups fcH- the interviews. Interviews with four groups of 
workshop participants (N=19) occuttk!. The evaluators used the following five phase process to 
select photographs and clarify or extend data derived from the oral interview. 

1. Hiotographs of training environments were serened individually and then collectively by 
the two internal project data collectors. 

2. Next, the two data collectors interviewed each other about perceptions invoked by the 
photographs of training. 

3. Photographs were assembled in preliminary structured units which documented a visual 
narrative. First , the photographic images were arranged by chronological experiences and 
as spatial maps of the training envirormienL 

4. Still photographs can be overwhelming in their volume of detail. To control for this, 
patterns were prioritized and judged to be pertinent in some way to the key informants who 
would be interviewed. In addition, training observations scales, benchmaric data, and 
summative questionnaire data were used to idendfy patterns. Capitalizing on assc ciations 
among these data sources, a resultant inventory was undertaken to select the most 
appropriate photographs to serve as interview stimuli. 
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5. Hie final phase involved abstracting and validating insights from photographs through 
detailed interviews with key informants. These oral interview questions, as well 
conversaticMialiy-related follow-up questions, reflected the evaluadon paradigm's six major 
quesdons. 

MAJOR LEARNINGS AND EMERGING POLEMICS 

An evaluation is only as good as the sh^ied Umweli or ineality which the evaluators are able 
to coalesce and make overt among all significant players. Cmainly, recording and interpreting 
pattmis should be within the scope of a good evaluation. Approaching this "willed" reality or new 
Umwelt can be facilitated by identijfying polemics as signposts. By presenting the following five 
polemics, we hope to stimulate discussion of how evaluating the impact of dynamic technology 
training projects can be approached and program improven^nt addressed. 

POLEMIC 1: INTERNAL VERSUS EXTERNAL ORIENTATION 

X 1— i ) —I ) I -J f > 

high Indwelling low Indwelling 

The reladonship among students, trainers, and photographers emerged with time. At first, we 
accepted the advice of Templine (1979) who emphasized the importance of shooting photographs 
based on elapsed time-intervals.in order to improve representativeness. Later, die nature of the 
photographic sampling changed over time due, at least in part, to the changing role of the 
photographers. As stated earlier, our role began as "clean" data collectors who regularly 
photographed every 20 minutes with two different lenses (a very wide angle lens to establish 
context and a normal or short telephoto lens to detail an aspect of the environment). There were 19 
or 20 minutes between shots in an intensive training environment Our impression was that much 
was being lost We discovered that the event sampling by tin^ resulted in "sterile" photos. When 
we added spontaneous incident photos to our event sampling strategy, we "crossed over" from 
the observer side of the continuum and became active participants, even to the point of serving as 
ancillary HyperCard tutors! 

In essence, we were directly experiencing what Michael Polanyi (1975) calls "indwelling" and 
the phenomenon of our tacit perceptions being made more oven and contextually sensitive as a 
result of extended personal dwelUng in a situation. As stated by Collier and Collier (1 986), "the 
production of photographs for purposes of research requires ... an overview of the culture or 
setting which allows all details to be seen within a context" This holistic view of a context can be 
achieved in a variety of ways, including following a participant as he or she passes through the 
program over a period of time fulfilling the material and social needs of that program. Thus the 
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photographer evaluaior will observe the culture of that program in functional association, 
meaningfully "ordered like the beads of a necklace" (Redficld, 1960). Accoidingly, the changes 
we experienced as we grew to be pan of the class culture resulted in more photographs which were 
able to better capture more critical events representadve of the conditions of the training 
environment. 

POl EMIC 2: HOLISM VERSUS DISRUmON 

f^- ) i ) f ^ ( ) ( ) 

holism disruption 

It was our premise that many photographs were better than too few when striving for a 
comprehensive picture of a program that captures multiple perceptions. Likewise, we felt diverse 
data collection methodologies were essential. Fa: example, we kept narrative logs (e.g., noting 
slide frames, date, time, topic and action) as well as collected context inventories, daily 
benchmarks, ar,d summative questionnaires fcr later analysis of the photos. 

The intrusive nature of photography demands that the "photographer evaluators" must stnif^le 
not to be distracting in order to reduce the commotion generated by their role. As things evolved 
naturally in this particular episode, we were persuaded to take a more participant approach to data 
collection. To do otherwise would have been would have been unnatural and awkward. This 
decision required "replotting die course" as a result of environmental variables. This was 
uncomfortable, particularly for the less experienced evaluates". 

Similarly, subjects must, as naturally as possible, make overt their successes and foibles. Air*; 
length of the coverage has much to do with the success of this occurring. The photographic 
literature is jammed with reports of master photographers who insist that time and a photographer's 
success in "capturing" subjects are correlated. The aggression and theft implied in the metaphor of 
"capturing" is not far from the truth. Intuitively, everyone seems to agree with Susan Sontag's idea 
that the most inclusive form of photographic acquisition is information. 

The photographic exolr^*^ ion and duplication of die world fragments continuities and feeds 
die pieces into an mtci.. ' '.e dossier, thereby providing possibilities of control that could 
not even be dreamed of under the earlier system of recording information: writing. (1 977. 
p. 156). 

As evaluators, we apprc iched our task aware of the potential for the subject to perceive the 
camera as an instrument which would single-out and expose their natural weakness in the learning 
environment. Given the number of pictures taken, many respondents reponed that they did not feel 
like they were "suigled" out and after the first night of shots, di ' not even notice die photographers 
presence for hours at a time. An interesting strategy for future photo-interviewing sessions would 
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be to include photographs taken by project participants themselves as Brown et.al. (1982) 
suggested in order to gather multiple perspectives of an event 

POLEMIC 3r VERBAL REALITY VERSUS VISUAL REALITY 

L 1 L I i 1 ( > ( \ 

highly visual highly verbal 

Photos appeared to act as both stimuli and verifiers of perception. Most interviewees 
indicated that the photographs helped "trigger" their memories and expand their perceptions of the 
training event The interviewers began with verbal question and verbal answer exchanges and 
discovered that the addition of photographs greatly added to the depth of the feedback. Whereas 
the verbal interview tended to elicit general perceptions, the photographs shared in small group 
settings elicited more specific perceptions, enabled detail«l probing and clarification of interviewee 
responses, and resulted in more viable suggestions for .project/training improvement. This was 
especially true of subjects finom visual arts disciplines who indicated they were more comfortable 
responding to visual stimuli. ^ 

'POLEMIC 4: PROVE VERSUS IMPROVE 

I ) i I L- \ i > ( ^ 

highly proof-oriented highly Improvement-oriented 

m 

To us, it appears photo-interviews can generate data particularly helpful for pilot-stage projc.cfs 
where innovation and development ratiier tiian status quo and maintenance are organizational 
priorities. Moreover, we posit that photo-interviewing can serve as an instructional as well as an 
evaluative strategy given the projective nature of visual images. Photographs prompt reflection 
upon the program being evaluated or reseaH:hed which goes beyond an interviewing situation 
lacking visual cues. Photographs seem to have a life of their own beyond the evaluation or the 
project. For example, those photographs which generated negative perceptions (e.g., viewed as 
inaccurate, disturbing or infuriating by informants) were often indicative of rejection of overt data 
at odds with their own internal data or belief structtires. Repeated exposure to diverse pictures 
enabled perceivcrs (both tiie evaluated and tiie evaluaiors) to develop a more accurate description of 
critical project contexts, processes and products. Armed with these holistic perceptions, it was 
possible to generated much more viable and negotiable suggestions for program improvement 
rather than being captured in a proof paradigm. 
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SUMMARY 

Overall, it appears to muld-modality inieiviewing facilitated the accuracy and quality of 
response. As a result of our investigation, we believe that photo-interviewing also yielded "richer 
data" than that usually obtained from verbal interviewing prcxedures alone. Informants tended to 
exanune images and react to cue:* present in those images more carefully than would have been 
expected using written or spoken cues alone. Photographs triggered recall and focused the 
interviewing process. Photogr^hic images solicited both differences and similariti?s in individual 
perception which can be analyzed across groups of i^rceivers. Photo- interviewing ; rovided a 
means of "getting inside" the program and its context to de^be and explain the program and its 
consequences in terms of participants' realities and meaning systems that oral interviewing did not 
permit 
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